Topics on Reduced Rank Methods for Multivariate Regression

Topics in Reduced Rank methods for Multivariate Regression by Ashin Mukherjee Advisors: Professor Ji Zhu and Professor Naisyin Wang Multivariate regression problems are a simple generalization of the univariate regression problem to the situation where we want to predict q(> 1) responses that depend on the same set of features or predictors. Problems of this type is encountered commonly in many quantitative fields. The main goal is to build more accurate and interpretable models that can exploit the dependence structure among the responses and achieve dimension reduction. Reduced rank regression has been an important tool to this end due to its simplicity, computational efficiency and superior predictive performance than much more complex models. In The first two parts of this thesis we investigate certain important practical aspects of the reduced rank regression method such as handling collinearity in the design matrix, selection of optimal rank. The last part focuses on extensions of the reduced rank methods to general functional models. In Chapter 2 we emphasize that the usual reduced rank regression is vulnerable to high collinearity among the predictor variables as that can seriously distort the singular structure of the signal matrix. To address this we propose the reduced rank ridge regression method that incorporates a ridge penalty in addition to the low rank constraint on the coefficient matrix. Ridge penalty introduces shrinkage which allows us to avoid the singularities when predictors are collinear. We are able to develop a straightforward computational algorithm to solve the optimization problem. We also discuss a novel extension of the reduced rank methodology to the Reproducing Kernel Hilbert Space(RKHS) setting.